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Abstract 

Background:  Voluntary  reporting  systems  identify  only  a  fraction  of  medical 
errors.  Electronic  identification  mechanisms,  which  are  more  efficient,  have  been 
defined  for  adverse  drug  events.  However,  similar  systems  are  lacking  for  other 
types  of  errors.  Objective:  The  investigators  sought  to  define  probabilistic 
strategies  that  could  support  quality  improvement  and  medical  error  detection  by 
decreasing  the  need  for  unselected  manual  chart  review.  Design:  Combinations  of 
administrative  data  and  laboratory  test  results  (“electronic  signatures”)  were 
employed  to  identify  discrete,  high-risk  clinical  situations  among  health  plan 
members  of  a  large  managed  care  organization.  The  design  used  was  a 
retrospective  cohort  study  linking  hospitalization  records,  outpatient  records,  and 
laboratory  results  that  were  formatted  using  approaches  developed  for  physiologic 
severity  scoring.  The  original  outcomes  of  interest  for  the  study  were  clinical 
situations  (e.g.,  birth  injuries  or  delayed  diagnosis  of  myocardial  infarction)  that 
have  a  strong  association  with  human  error.  Results:  When  presented  with 
preliminary  results,  senior  leaders  in  the  investigators’  parent  organizations  raised 
a  number  of  objections  to  any  public  presentation  or  publication  of  the  results. 
Because  of  these  objections,  the  quantitative  results  presented  in  this  report  focus 
on  rapid  detection  of  one  outcome — prolonged  neonatal  assisted  ventilation — that 
has  a  weak  association  with  human  error.  Using  recursive  partitioning,  the 
investigators  were  able  to  define  subsets  of  newborns  for  whom  the  frequency  of 
the  outcome  of  interest  was  substantially  higher  than  in  the  general  population 
(1  percent).  For  example,  an  electronic  signature  identified  a  subset  of  infants 
(comprising  4  percent  of  the  birth  cohort)  in  which  the  outcome  of  interest 
occurred  in  22  percent  of  the  newborns.  Conclusions:  Use  of  probabilistic 
electronic  strategies  could  yield  significant  benefits  in  medical  error  research  as 
well  as  major  operational  improvements  in  medical  error  detection  and  reporting, 
quality  assurance,  and  quality  improvement.  However,  three  barriers  are  likely  to 
limit  the  use  of  such  “electronic  signatures” — fear  of  malpractice  litigation,  fear 
of  lawsuits  invoking  “enterprise  liability,”  and  high  development  costs.  Entities 
most  likely  to  benefit  from  these  approaches  are  those  with  a  critical  mass  of 
experienced  personnel,  a  circumstance  that  can  spread  the  development  costs  over 
a  large  number  of  hospitals  and/or  clinics. 
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Introduction 

Considerable  public  attention  has  been  devoted  to  whether  reporting  of  human 
error  in  medicine  should  be  mandatory  or  voluntary.  Less  attention  has  been 
given  to  the  problems  of  implementing  error-detection  and  error-reporting 
systems  in  health  care  organizations.  In  this  report,  we  describe  the  problems 
encountered  when  we  developed  and  attempted  to  report  a  probabilistic  electronic 
scanning  mechanism  in  a  large  managed  care  organization,  the  Northern 
California  Region  of  the  Kaiser  Permanente  Medical  Care  Program  (KPMCP), 
which  consists  of  three  corporations  representing  the  program’s  doctors  (The 
Pennanente  Medical  Group,  Inc.),  hospitals  and  clinics  (Kaiser  Foundation 
Hospitals,  Inc.),  and  insurance  plan  (Kaiser  Foundation  Health  Plan,  Inc.) 
operating  under  a  mutual  exclusivity  arrangement. 

Most  hospitals  rely  on  three  mechanisms  for  error  detection — manual  chart 
review,  which  often  involves  a  clinical  department  examining  all  eligible  patient 
charts;  voluntary  reporting  (often  referred  to  as  “incident  reporting”);  and  chart 
reviews  combined  with  clinician  interviews  following  inquiries  from  the  legal 
system.  These  mechanisms  have  been  shown  to  be  inferior  to  structured 
approaches,4' 5  with  a  major  problem  being  the  reluctance  of  individuals  to  report 
incidents.6  Current  error-reporting  mechanisms  also  have  another  major 
limitation — timeliness,  with  notification  time  lags  as  long  as  several  years  from 
when  an  event  occurred. 

One  of  the  best  examples  that  demonstrates  the  potential  of  electronic 
scanning  is  a  study  that  electronically  captured  adverse  drug  events  (ADEs) 
among  36,653  hospitalized  patients  over  an  18-month  period.  Bates  et  al.7 
employed  a  probabilistic  approach  that  relied  on  the  fact  that  ADEs  are  often 
associated  with  distortions  in  the  usual  pattern  of  electronic  infonnation  (i.e., 
“signals”)  generated  by  the  process  of  care.  These  researchers  developed 
computer  algorithms  to  scan  their  hospital’s  computer  systems  for  specific 
patterns — which  we  call  “electronic  signatures” — that  were  associated  with  ADEs 
(e.g.,  reports  of  elevated  drug  levels  or  of  unusual  drug  or  test  combinations). 
Using  several  different  methods,  they  verified  731  ADEs  out  of  a  total  of  557,860 
drug  exposures  in  their  cohort.  Of  these  events,  only  nine  (1.2  percent)  were 
identified  using  traditional  incident  reports.  In  contrast,  electronic  scanning 
identified  63 1  (86  percent)  of  the  ADEs,  while  the  remaining  cases  were 
identified  by  voluntary  reporting  following  electronic  prompting. 

The  purpose  of  our  study  was  to  expand  on  the  work  of  Bates  et  al.  in  three 
ways.  First,  we  sought  to  use  electronic  scanning  for  the  identification  of  discrete 
clinical  events  other  than  ADEs.  Medication  errors  are  important,  but  they  are  not 
the  only  kind  of  error  in  medicine.  High  capture  rates  of  other  types  of  errors  can 
be  achieved  using  rigorous  chart  review  protocols,4' 5  but  it  would  be  useful  to 
define  strategies  that  minimize  manual  chart  review.  Second,  we  employed 
electronic  scanning  based  on  two  commonly  available  infonnation  systems:  an 
admission-discharge-transfer  (ADT)  hospitalization  database  and  a  laboratory 
database,  both  of  which  are  in  common  use.  This  differs  somewhat  from  the 
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approach  used  by  Bates  et  al.7  in  that  their  algorithms  took  advantage  of  a  hospital 
with  unusually  sophisticated  information  systems.  Finally,  unlike  Bates  et  al.’s 
single-center  study,  our  electronic  scanning  algorithms  were  designed  for  use 
across  an  entire  hospital-and-clinic  system  consisting  of  16  hospitals  and  34 
clinics  that  employed  the  same  information  systems. 


Design  and  results 

Our  original  goal  was  to  define  “electronic  signatures”  for  clinical  events  that 
could  have  a  strong  association  with  medical  error,  such  as  birth  injuries  or 
delayed  diagnosis  of  myocardial  infarction.  However,  when  we  presented  our 
preliminary  results  to  senior  KPMCP  leaders,  they  raised  a  number  of  significant 
institutional  concerns  that  are  the  primary  focus  of  this  report.  The  first  concern 
was  that  any  report  involving  public  presentation  or  publication  in  the  peer- 
reviewed  literature  should  be  restricted  to  outcomes  having  only  a  weak 
association  with  human  error.  Because  of  this  concern,  the  quantitative 
component  of  this  report  focuses  on  the  occurrence  of  neonatal  length  of  assisted 
ventilation  (LOAV)  >  5  days,  an  outcome  which  is  rarely  of  medical-legal 
concern.  This  is  because  many  newborns,  particularly  premature  infants  or  term 
infants  treated  for  sepsis,  pulmonary  hypertension,  or  congenital  anomalies, 
experience  prolonged  assisted  ventilation  without  any  error  having  occurred.  The 
data  for  LOAV  >  5  days  is  provided  as  background  for  a  description  of  the 
interactions  we  had  with  our  organizational  leadership.  The  problems  we 
encountered,  and  the  solutions  we  adopted,  have  significant  implications  for  other 
investigators  seeking  to  improve  error  detection,  error  reduction,  and  quality 
improvement  processes. 

Our  conclusions  regarding  the  implications  of  our  approach  are  based  on 
extensive  conversations  and  correspondence  with  several  individuals.  These 
involved  senior  leaders,  including  legal  counsel,  of  the  KPMCP,  and  program 
staff  at  the  National  Patient  Safety  Foundation,  one  of  the  entities  funding  this 
research. 

Development  of  electronic  signatures 

The  electronic  signatures  described  in  this  report  for  identifying  babies  with 
LOAV  >  5  days  were  selected  on  the  basis  of  potential  usefulness  to  a 
hypothetical  obstetrics  or  neonatology  department  that  uses  traditional  screening 
mechanisms  (i.e.,  those  involving  review  of  100%  of  obstetric  or  neonatal  charts). 
We  are  reporting  on  signatures  that  achieved  the  maximum  degree  of 
“enrichment.”  For  example,  if  a  department  ordinarily  reviewed  1,000  charts  to 
identify  12  events,  it  would  be  useful  to  identify  a  reduced  group  of  charts,  which 
was  “enriched”  by  a  factor  of  10  (e.g.,  a  subset  of  75  records,  9  of  which  had  the 
outcome  of  interest). 

We  developed  electronic  signatures  using  a  retrospective  cohort  study  design, 
using  linked  electronic  records  from  five  KPMCP  hospitals.  The  study  was 
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approved  by  the  KPMCP  Institutional  Review  Board  for  the  Protection  of  Human 
Subjects.  Eligible  study  subjects  were  (1)  all  women  who  delivered  a  live-bom 
infant  during  a  3 -year  period  and  (2)  all  babies  bom  alive  to  these  women. 

Inferences  regarding  application  of  these  methods  to  other  situations  are  based 
on  consideration  of  the  following  themes  in  the  medical  literature:  common,  well- 
described  physiologic  processes  (e.g.,  the  known  rise  of  certain  enzymes 
following  myocardial  infarction),  probabilistic  approaches  based  on  the 
relationship  between  errors  and  adverse  patient  outcomes,  specific  descriptions 
of  error  detection  and  reduction  efforts,13  15  and  the  superiority  of  process  (as 
opposed  to  outcomes)  measures  for  detecting  problems  in  the  quality  of  care.16 

Using  methods  described  elsewhere,17-21  we  identified  and  linked  all  eligible 
maternal  hospitalization  records,  maternal  outpatient  visit  records,  neonatal 
hospitalization  records,  and  specific  components  (PaCCF,  PaCfi,  FiC>2,  and  base 
deficit)  of  neonatal  arterial  blood  gas  measurements  obtained  during  the  first  24 
hours  of  age.  We  defined  22  candidate  predictor  variables,  which  were  of  two 
types:  raw  (i.e.,  laboratory  data  that  were  not  transformed  in  any  way)  and  derived 
(i.e.,  laboratory  results  that  were  transfonned  into  scores  or  indices).  The 
approach  used  to  create  derived  predictor  variables  was  based  on  the  development 
of  the  Score  for  Neonatal  Acute  Physiology  (SNAP),  Versions  I  and  II.  Table 
1  provides  examples  of  the  candidate  predictor  variables.  Interested  readers  can 
obtain  a  complete  list  of  predictors  from  Dr.  Escobar. 

Because  all  of  the  predictor  variables  that  we  employed  were  highly  correlated 
with  positive  or  negative  correlation  coefficients  (absolute  value  exceeding  0.5), 
we  elected  to  develop  our  models  using  a  nonparametric  technique  known  as 
recursive  partitioning.  Recursive  partitioning  analyses  were  perfonned  using 
Classification  and  Regression  Trees  (CART)  software.25' 26  The  CART  software 
we  employed  uses  one-fourth  to  one-third  of  a  dataset  as  a  “learning”  dataset  and 
then  generates  a  validation  outcome  tree  with  terminal  nodes.  These  nodes  are 
clusters  of  records  that  meet  specific  criteria,  and  thus  are  enriched  with  the 
outcome  of  interest.  For  simplicity,  we  are  reporting  our  results  as  extrapolated  to 
the  entire  dataset,  not  just  to  the  validation  dataset.  Also,  in-hospital  deaths  of 
neonates  have  been  grouped  with  those  infants  with  LOAV  >  5  days,  making  the 
assumption  that  newborns  who  died  in  the  neonatal  period  would  have  required 
prolonged  assisted  ventilation  had  they  survived. 

Findings 

During  the  study  period,  a  total  of  41,439  deliveries  occurred  at  the  study 
sites,  with  a  total  of  42,1 10  live  births.  Table  2  summarizes  data  from  this 
population.  Figure  1  shows  the  results  of  recursive  partitioning  applied  to  the 
subset  of  40,490  babies  bom  at  36  weeks  or  more  of  gestation,  a  group  in  which 
babies  who  experience  prolonged  assisted  ventilation  are  very  rare  (only 
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Figure  1.  Results  of  recursive  partitioning  among  infants  born  >  36  weeks  gestation 
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0.4  percent  of  babies).  An  electronic  signature  based  on  one  of  the  oxygenation 
components  of  the  SNAP,  ~  and  on  the  failure  to  oxygenate  (FTOX)  variable 
described  in  Table  1,  achieved  a  twentyfold  enrichment  in  prolonged  assisted 
ventilation  (i.e.,  the  proportion  with  the  outcome  of  interest  rose  from  0.4  to  8.0 
percent).  Applied  to  the  entire  cohort  of  42,1 10  births,  the  signature  shown  in 
Figure  1  identified  a  subset  of  1,724  infants  (4  percent)  in  whom  the  rate  of 
LOAV  >  5  days  was  22  percent  (in  contrast  to  1  percent  for  the  entire  cohort). 

Figure  2  shows  the  results  of  a  different  electronic  signature,  based  on  the 
relationship  between  maternal  triage  in  the  labor  and  delivery  service  and  the 
occurrence  of  LOAV  >  5  days  among  deliveries  occurring  at  less  than  36  and  at 
least  36  weeks  of  gestation.  It  shows  that  women  who  deliver  at  36  or  more  weeks 
of  gestation  and  who  were  “triaged”  home  (i.e.,  who  have  to  return  for  labor  and 
delivery)  are  more  likely  to  deliver  a  newborn  with  LOAV  >  5  days.  In  contrast, 
no  such  relationship  was  evident  among  deliveries  occurring  at  less  than  36 
weeks. 

Other  possible  uses  of  this  approach 

Table  3  summarizes  five  possible  electronic  signatures  for  important  clinical 
situations.  Each  of  these  signatures  is  based  upon  commonly  available 
information  systems  and  plausible  clinical  scenarios.  For  example,  situation  #3 
involves  patients  who  undergo  routine  surgical  procedures  or  who  deliver  an 
infant.  Such  patients  ordinarily  do  not  require  arterial  blood  gas  measurement  and, 
in  these  cases,  after  removing  patients  with  known  pre-existing  comorbidities,  the 
simple  fact  that  an  arterial  blood  gas  measurement  was  obtained — rather  than  any 
particular  value  of  the  pH  or  Pa02 — would  be  indicative  of  a  greater  chance  of 
error. 


Interactions  with  our  parent  organizations 

The  reactions  we  encountered  from  our  parent  organizations  were  mixed. 
Some  quality  and  risk  managers  were  extremely  enthusiastic  about  transplanting 
these  techniques  from  the  research  environment  into  the  operational  arena.  Others 
were  uncomfortable  with  that  possibility  because,  given  limited  resources  and 
competing  priorities,  it  would  be  difficult  to  incorporate  these  techniques  into 
existing  operations. 

While  some  senior  leaders  were  enthusiastic  about  conducting  research  on 
electronic  signatures  for  purely  internal  purposes  (i.e.,  within  the  protected 
environment  of  quality  management  bodies),  considerable  discomfort  was 
expressed  with  the  notion  of  conducting  research  and  publishing  it  in  the  public 
domain.  Our  initial  reaction  to  this  was  one  of  dismay.  At  first,  it  seemed  unlikely 
that  presenting  or  publishing  data  involving  the  electronic  signatures  shown  in 
Table  3  could  present  a  risk  to  any  organization  or  individual.  None  of  these 
signatures,  by  themselves,  constitute  proof  that  an  error  leading  to  an  adverse 
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Figure  2.  Relationship  between  maternal  triage  in  labor  and  delivery  and  the  occurrence  of  prolonged  assisted  ventilation 
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outcome  or  a  “near  miss”  had  occurred.  Plausible  clinical  scenarios  exist  that 
could  be  associated  with  the  presence  of  any  of  these  patterns. 

However,  upon  further  reflection  and  consultation,  we  realized  that  fears 
about  publishing  such  data  in  the  public  domain  have  a  rational  basis.  We  suspect 
that  these  fears  and  concerns  are  shared  by  other  organizations — even  those  with 
lower  profiles  in  their  communities  than  the  KCMCP — and  are  acting  as  strong 
disincentives  to  conduct  human  error  research  throughout  medicine.  The  major 
concerns  raised  were  the  following: 

1 .  Being  the  first  to  report  rates  of  errors — or  even  proxies  for  errors — 
would  be  damaging  from  a  public  relations  standpoint  in  those 
situations  for  which  there  are  no  published  comparison  data,  so  that 
even  a  low  rate  would  look  bad  to  the  public. 

2.  Results  of  an  electronic  scanning  project  might  be  subpoenaed  by  a 
plaintiffs  attorney  and  lead  to  identification  of  other  patients  who 
became  critically  ill,  subjecting  the  organization  to  further  litigation.  It 
should  be  noted  that,  in  the  United  States,  the  statute  of  limitations  for 
some  specialties,  like  obstetrics,  can  be  considered,  for  practical 

27  28 

purposes,  virtually  limitless  due  to  a  wide  variety  of  legal  theories.  ’ 

3.  Results  of  electronic  scanning  might  be  subpoenaed  and  used  in  court 
as  evidence  of  a  pattern  of  inappropriate  care  within  an  institution,  an 
assertion  that  could  be  employed  in  a  number  of  situations. 

4.  Publication  of  a  paper  describing  results  of  electronic  scanning  might 
lead  plaintiffs’  attorneys  to  seek  out  other  families  or  patients  who 
experienced  an  adverse  outcome  during  the  time  period  and  at  the 
locations  reported  in  a  scientific  publication. 

5.  Some  clinicians  were  also  very  worried  about  being  “judged”  on  the 
basis  of  purely  electronically  collected  data  (i.e.,  that  electronic 
signatures  could  become  de  facto  measurement  standards).  Clinicians 
want  an  impartial  and  scientific  form  of  peer  review  to  assess  whether 
an  avoidable  error  did  in  fact  occur. 

6.  Because  electronic  scanning  is  more  efficient  than  manual  chart 
review,  the  number  of  cases  brought  for  review  by  quality 
management  bodies  could  increase  substantially.  Given  limited 
resources,  these  bodies  might  not  be  able  to  handle  the  increased  load. 

After  extensive  discussions  and  consultations,  a  compromise  strategy  to 
overcome  these  objections  was  agreed  upon.  For  internal  purposes,  we  employed 
the  methods  described  here  to  support  KPMCP  projects  whose  measurement 
components  reside  within  protected  domains,  such  as  facility  or  regional  peer 
review  bodies.  For  purposes  of  communicating  with  the  scientific  community,  we 
selected  the  outcome  reported  here — LOAV  >  5  days — as  an  example  of  a  serious 
clinical  situation  that,  although  significant  in  tenns  of  a  patient’s  illness  severity, 
has  a  weak  association  with  human  error. 
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Discussion 

We  have  shown  that  it  is  possible  to  employ  arterial  blood-gas  results  and 
triage  patterns  to  define  electronic  signatures  associated  with  the  occurrence  of  a 
discrete  clinical  event  (i.e.,  occurrence  of  prolonged,  assisted  ventilation  lasting  at 
least  5  days  during  the  initial  neonatal  period).  Use  of  this  approach  could  be 
extended  to  situations  that  are  more  likely  to  have  a  high  association  with  the 
occurrence  of  human  error,  such  as  those  shown  on  Table  3.  In  busy  hospitals 
using  older  methods,  the  effects  of  switching  to  probabilistic  approaches  could  be 
substantial.  For  example,  a  neonatology  director  employing  manual  chart  review 
to  identify  babies  with  long  ventilator  courses  in  a  hospital  with  3,000  deliveries 
per  year  could  reframe  his  or  her  search  strategies.  One  option  is  to  use  traditional 
approaches,  reviewing  250  charts  per  month  to  find  all  eligible  charts.  Another 
option  is  to  use  electronic  scanning,  reviewing  10  charts  a  month  to  find  two 
problem  charts  very  rapidly  (the  signature  we  developed  is  based  on  data  elements 
available  within  24  hours  after  birth,  which  makes  them  more  timely  than  data 
generated  at  the  time  of  discharge,  such  as  billing  data).  In  the  usual  course  of 
scientific  inquiry,  the  next  logical  steps  would  be  to  test  such  signatures  by 
scanning  data  from  an  eligible  population,  defining  their  sensitivity  and 
specificity,  and  calculating  the  area  under  their  receiver-operator  characteristic 
curves,  and  then  to  present  these  findings  to  the  scientific  community  for  further 
discussion,  validation,  and/or  refutation.  We  have,  in  fact,  refined  our  work  on  the 
prediction  of  assisted  ventilation  and  have  submitted  this  for  publication 
elsewhere.29 

In  the  case  of  adverse  drug  events,  some  peer-reviewed  publications  have 
been  fairly  specific  with  respect  to  delineating  times  and  locations,  sometimes  to 
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the  point  that  a  reader  could  pinpoint  the  specific  hospital  ward  involved.  ’ 
Nonetheless,  just  a  few  years  later,  and  despite  a  significant  increase  in  both 
public  attention  to  human  error  and  funding  for  patient  safety  research,  large 
health  care  institutions  are  still  uncomfortable  with  the  publication  of  research 
findings  in  this  area.  What  has  changed? 

We  suspect,  to  quote  Davidoff,33  there  is  another  “elephant  in  the  room” — 
enterprise  liability34  37 — and  that  its  impact  may  differ  from  what  Kohn 
anticipated  in  the  Institute  of  Medicine  report.38  The  term  “enterprise  liability” 
itself  needs  clarification  in  the  context  of  present  efforts  to  reduce  medical  error 
and  assuage  provider  concerns  about  malpractice  claims.39  For  our  purposes, 
“enterprise  liability”  refers  to  various  circumstances  when  an  organization,  for 
instance  a  hospital,  health  maintenance  organization,  or  health  plan,  is  potentially 
liable  to  injured  patients.  In  one  situation,  the  hospital  might  be  liable  for  failure 
to  fulfill  its  obligation  to  patients,  for  instance,  by  not  providing  appropriate  staff 
in  the  emergency  room.  In  another  situation,  a  hospital  might  be  held  legally 
responsible  for  the  injuries  caused  by  a  nonemployee,  such  an  independent 
contractor  surgeon.  Fear  of  litigation  aimed  at  establishing  some  form  of 
organizational  liability  is  likely  to  exceed  fear  of  malpractice  litigation  aimed  at 
physicians  and  to  deter  quantitative  research  in  the  area  of  human  error. 
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Fear  of  enterprise  liability  could  make  perfonning  research  on  systematic 
human  error  and  reporting  the  findings  potentially  risky  to  health  care 
organizations.  Consider  this  hypothetical  scenario:  A  physician-researcher 
employed  by  a  university  desires  to  perform  research  using  data  and  information 
from  affiliated  hospitals  and  their  feeder  clinics.  The  researcher  proposes  to 
employ  institutional  databases  to  assess  a  new  quality  marker  (e.g.,  ratio  of  time 
to  diagnosis  of  a  particular  disease  state  on  the  weekend  to  time  to  diagnosis  on  a 
weekday).  The  corporation  feels  that  the  project  is  of  theoretical  interest  and 
might  conceivably  provide  some  benefits,  and  so  gives  permission  for  the 
researcher  to  submit  a  proposal  for  a  Federal  research  grant.  After  the  researcher 
obtains  funding  and  approval  by  the  organizational  Institutional  Review  Board, 
the  appropriate  corporate  officer  grants  access  to  the  databases  required  to 
conduct  the  study. 

The  study  is  performed,  and  its  results  indicate  that  significant  variation  exists 
across  the  feeder  clinics  for  the  new  quality  marker.  One  study  conclusion  is  that, 
due  to  delays  in  diagnosis,  not  all  patients  with  the  relevant  diagnosis  appear  to 
have  an  equal  chance  of  being  referred  to  entity  hospitals  with  appropriate  clinical 
facilities  for  treatment.  Although  the  publication  does  not  make  strong  inferences 
about  the  possible  clinical  consequences,  a  reasonably  careful  reader  might  find 
some  significant  ramifications.  The  results  are  reported;  the  sites  of  the  study  are 
not  named,  but  the  state  and  affiliation  of  the  researcher  are.  The  researcher  notes 
that  although  the  quality  indicators  obtained  do  not  suggest  poor  care,  no 
inferences  are  possible  due  to  the  newness  of  the  indicator  and  the  lack  of 
comparative  data  in  the  literature. 

Some  time  later,  an  attorney  for  an  individual  whose  diagnosis  of  the  above 
disease  state  was  delayed  hears  of  the  researcher’s  study  through  a  presentation  at 
a  scientific  conference.  The  attorney  makes  the  correct  inference  that  the  sites 
described  in  the  researcher’s  presentation  include  the  one  where  his  client 
received  care.  The  attorney  sues  the  hospital  corporation  for  two  reasons:  First,  it 
is  a  “deeper  pocket”  than  an  individual  provider,  and  thus  amenable  to  greater 
damage  judgments.  Second,  by  suing  the  corporate  entity,  the  attorney  aims  to 
employ  the  legal  discovery  process  to  obtain  all  information  regarding  the 
researcher’s  data  as  well  as  corporate  documents. 

These  documents  would  be  desirable  to  the  attorney  because  they  could 
provide  persuasive  evidence  to  a  jury  that  (1)  the  corporate  entity  approved  the 
study,  as  evidenced  by  its  Institutional  Review  Board  approval  and  its  officer 
granting  access  to  organizational  databases;  (2)  the  hospital  corporation  knew — or 
should  have  known — about  the  high  level  of  variation  between  facilities  it  owned; 
and  (3)  the  corporate  entity  failed  to  address  the  problems  which  were  implied  by 
the  researcher’s  study.  Importantly,  neither  the  corporate  documents  nor  the 
researcher’s  data  would  be  protected  by  any  legal  shields  such  as  peer  review/ 
quality  assurance  privilege  or  attorney-client  privilege.40  42  In  fact,  even  root 
cause  analyses  and  corrective  action  plans  have  been  deemed  discoverable.43'44 

Such  a  sequence  of  events,  plausible  under  the  U.S.  legal  system  and  which 
could  also  include  the  possibility  of  plaintiff  bias,44  would  tend  to  chill 


63 


Advances  in  Patient  Safety:  Vol.  3 


institutional  promotion  of  human  error  research,  or,  at  the  very  least,  publication 
of  its  findings.  Several  implications  are  likely.  First,  since  it  is  well  known  that 
existing  mechanisms  underestimate  the  incidence  of  errors,45  an  organizational 
entity  that  considers  making  well-structured  efforts  to  detect  errors  will 
necessarily  need  to  take  a  broader  view  of  the  costs  of  the  error-detection  process. 
The  organization  will  need  to  consider  not  just  the  direct  costs  of  error  detection, 
but  also  the  costs  of  implementing  corrective  action,  the  potential  costs  of  legal 
exposure,  and  the  effects  on  public  relations  should  litigation  occur.  Second, 
corporate  entities  are  likely  to  require  the  use  of  de-identification  mechanisms 
such  as  those  strongly  recommended  by  the  Institute  of  Medicine’s  Report,  To  Err 
Is  Human.  A  similar  approach,  using  pooled  datasets,  has  become  routine  in 
aviation,46  and  some  de-identification  has  been  used  in  medical  research.47  49 
However,  de-identification  may  not  be  a  viable  alternative  for  individual  entities 
(as  opposed  to  consortia  that  pool  data)  interested  in  promoting  patient  safety 
research.  Consequently,  both  cost  and  confidentiality  concerns  would  tend  to 
make  research  into  medical  error  more  difficult  and  expensive.43' 50 

Indeed,  some  authors  have  already  noted  that  attempts  to  gather  more  data  as 
part  of  the  efforts  to  enhance  patient  safety  are  “on  a  collision  course  with  the 
medical  malpractice  system.”51  Their  preferred  solution — no-fault  compensation 
for  medical  injuries — is  not  likely  to  be  considered  by  policymakers  until  the 
notion  of  enterprise  liability  is  addressed  as  part  of  a  system  of  public 
accountability  for  error  prevention.  Furthermore,  the  logic  of  a  systems 
approach  to  patient  safety  is  to  look  at  organizational,  as  opposed  to  individual, 
practices  as  a  means  of  error  prevention  or  reduction.  The  proponents  of  a  no-fault 
system  of  compensation  recognize  the  changing  face  of  health  care  financing  and 
delivery.  They  point  to  organizations  or  enterprises,  not  individual  physicians  or 
their  insurers,  being  the  source  of  compensation.51  This  is  consistent  with  some 
calls  for  reformulation  of  ethical  precepts  suitable  for  system  accountability. 

At  least  one  author  has  suggested  that  the  tenn  “vicarious  liability”  should  be 
used  instead  of  “enterprise  liability”  to  remind  us  of  the  potential  that  liability 
rules  have  to  act  as  incentives  for  organizations  to  improve  the  quality  of  care. 
Such  improvements  could  take  place  through  new  kinds  of  contractual 
arrangements  with  physicians  and  other  professionals.  Organizations,  as 
opposed  to  individual  professionals,  are  in  the  position  to  use  newer  approaches  to 
error  reduction,  such  as  the  ones  suggested  by  our  study.  Organizations  must 
understand  the  risks,  not  only  to  themselves  as  corporate  entities,  but  also  to  the 
quality  of  care.  Furthermore,  they  must  understand  the  possible  liability  risks  of 
not  undertaking  new  studies  to  reduce  medical  error.54  Perhaps  a  day  will  come 
when  corporations  will  increase  support  for  human  error  research  and  human 
error  reduction,  because  having  corrective  processes  in  place  may  not  only  be 
better  business  practice,  but  also  provide  protection  from  enterprise  liability.55 

While  it  is  desirable  to  have  health  care  organizations  begin  to  use 
probabilistic  approaches  such  as  described  in  this  report,  two  other  practical 
considerations  must  be  kept  in  mind  by  policymakers  wishing  to  increase  patient 
safety  research  and  societal  attention  to  patient  safety. 
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The  first  consideration  is  that  probabilistic  approaches  to  error  detection  are 
most  likely  to  be  employed  by  large  corporate  entities.  The  costs  of  developing 
and  validating  electronic  error  detection  mechanisms  are  high  and,  with  the 
exception  of  a  few  stand-alone  hospitals  that  have  experience  in  attracting 
research  funding,  it  is  unlikely  that  smaller  institutions  will  be  able  to  afford  the 
high  development  costs.  This  is  particularly  the  case  in  the  context  of  smaller 
institutions’  historical  financial  weakness  and  need  to  address  other  legal 
mandates.43 

The  second  consideration  is  that  entities  wishing  to  employ  these  techniques 
for  internal  quality  assurance  and  improvement  must  consider  not  only 
enterprise/organizational  liability,  but  also  the  specific  provisions  of  the  laws 
relating  to  the  legal  discovery  process  in  the  jurisdictions  in  which  they  operate. 
In  the  United  States,  there  has  been  a  general  trend  towards  decreasing  legal 
protections  of  the  internal  peer  review  process.56  Limited  protection  is  extant  in 
some  circumstances,  but  the  specific  legal  rules  vary  by  jurisdiction,  and 
researchers  must  pay  meticulous  attention  to  the  law  as  it  applies  to  them  at  the 
location  where  they  are  working.38’ 56  For  example,  in  some  jurisdictions,  even 
casual  sharing  of  data  from  a  peer  review  committee  to  others  internally  in  the 
“normal  course  of  business”  could  lead  to  loss  of  legal  protection  of  that 
information  and  thus  disclosure  of  that  sensitive  information.56 


Conclusion 

Use  of  electronic  scanning  of  medical  records  for  detection  of  human  error  is 
a  viable  approach  for  both  research  and  quality  improvement.  However, 
implementation  of  this  approach  requires  that  individuals  and  organizations 
consider  a  number  of  issues  that  have  received  relatively  little  attention  in  the 
literature.  One  issue  is  the  need  to  separate  activities  into  two  domains:  those 
conducted  under  peer  review  protection  and  those  conducted  for  publication  or 
dissemination.  A  second  issue  is  that  a  broader  cost-  and  risk-benefit  assessment 
needs  to  be  performed  prior  to  incorporating  this  approach  into  operations. 
Finally,  significant  advances  in  the  use  of  electronic  scanning  are  unlikely  to 
occur  unless  researchers  and  practitioners  take  due  consideration  of  enterprise 
liability  and  of  the  need  to  employ  de-identification  mechanisms. 
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